Multi-stage recognition of named entities in natural language text based on morphological and semantic features

ABSTRACT

Systems and methods for multi-stage recognition of named entities based on morphological and semantic features of natural language texts. An example method comprises: performing a lexico-morphological analysis of a natural language text comprising a plurality of tokens, each token comprising at least one natural language word; determining, based on the lexico-morphological analysis, one or more lexical meanings and grammatical meanings associated with each token of the plurality of tokens; for each token the plurality of tokens, evaluating one or more classifier functions using the lexical and grammatical meanings associated with the tokens, wherein a value of each classifier function is indicative of a degree of association of the token with a category of named entities; performing a syntactico-semantic analysis of at least part of the natural language text to produce a plurality of semantic structures representing the part of the natural language text; and interpreting the semantic structures using a set of production rules to determine, for one or more tokens comprised by the part of the natural language text, a degree of association of the token with a category of named entities.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of priority under 35 USC 119to Russian patent application No. 2016124139, filed Jun. 17, 2016; thedisclosure of which is herein incorporated by reference in its entiretyfor all purposes.

TECHNICAL FIELD

The present disclosure is generally related to extracting informationfrom natural language texts executed by computer systems, and is morespecifically related to multi-stage recognition of named entities basedon morphological and semantic features.

BACKGROUND

Information extraction is one of the important operations in automatedprocessing of natural language texts. Named-entity recognition (NER)(also known as entity identification, entity chunking and entityextraction) is an information extraction task that locates andclassifies tokens in a natural language text into pre-defined categoriessuch as names of persons, organizations, locations, expressions oftimes, quantities, monetary values, percentages, etc.

SUMMARY OF THE DISCLOSURE

In accordance with one or more aspects of the present disclosure, anexample method for multi-stage recognition of named entities in naturallanguage texts based on morphological and semantic features maycomprise: performing, by a computer system, a lexico-morphologicalanalysis of a natural language text comprising a plurality of tokens,each token comprising at least one natural language word; determining,based on the lexico-morphological analysis, one or more lexical meaningsand grammatical meanings associated with each token of the plurality oftokens; for each token the plurality of tokens, evaluating one or moreclassifier functions using the lexical and grammatical meaningsassociated with the tokens, wherein a value of each classifier functionis indicative of a degree of association of the token with a category ofnamed entities; performing a syntactico-semantic analysis of at leastpart of the natural language text to produce a plurality of semanticstructures representing the part of the natural language text; andinterpreting the semantic structures using a set of production rules todetermine, for one or more tokens comprised by the part of the naturallanguage text, a degree of association of the token with a category ofnamed entities.

In accordance with one or more aspects of the present disclosure, anexample system for multi-stage recognition of named entities in naturallanguage texts based on morphological and semantic features maycomprise: a memory and a processor, coupled to the memory, the processorconfigured to: perform a lexico-morphological analysis of a naturallanguage text comprising a plurality of tokens, each token comprising atleast one natural language word; determine, based on thelexico-morphological analysis, one or more lexical meanings andgrammatical meanings associated with each token of the plurality oftokens; for each token the plurality of tokens, evaluate one or moreclassifier functions using the lexical and grammatical meaningsassociated with the tokens, wherein a value of each classifier functionis indicative of a degree of association of the token with a category ofnamed entities; perform a syntactico-semantic analysis of at least partof the natural language text to produce a plurality of semanticstructures representing the part of the natural language text; andinterpret the semantic structures using a set of production rules todetermine, for one or more tokens comprised by the part of the naturallanguage text, a degree of association of the token with a category ofnamed entities.

In accordance with one or more aspects of the present disclosure, anexample computer-readable non-transitory storage medium may compriseexecutable instructions that, when executed by a computer system, causethe computer system to: perform a lexico-morphological analysis of anatural language text comprising a plurality of tokens, each tokencomprising at least one natural language word; determine, based on thelexico-morphological analysis, one or more lexical meanings andgrammatical meanings associated with each token of the plurality oftokens; for each token the plurality of tokens, evaluate one or moreclassifier functions using the lexical and grammatical meaningsassociated with the tokens, wherein a value of each classifier functionis indicative of a degree of association of the token with a category ofnamed entities; perform a syntactico-semantic analysis of at least partof the natural language text to produce a plurality of semanticstructures representing the part of the natural language text; andinterpret the semantic structures using a set of production rules todetermine, for one or more tokens comprised by the part of the naturallanguage text, a degree of association of the token with a category ofnamed entities.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of examples, and not by wayof limitation, and may be more fully understood with references to thefollowing detailed description when considered in connection with thefigures, in which:

FIG. 1 depicts a flow diagram of an example method for multi-stagerecognition of named entities in natural language texts based onmorphological and semantic features, in accordance with one or moreaspects of the present disclosure;

FIG. 2 schematically illustrates an example graphical user interface(GUI) that may be employed for accepting user input specifying namedentity categories for certain tokens of a representative fragment of anatural language text, in accordance with one or more aspects of thepresent disclosure;

FIG. 3 schematically illustrates an example of a marked up naturallanguage text that may be employed for training a classifier thatproduces a degree of association of a token of a natural language textwith a certain category of named entities, in accordance with one ormore aspects of the present disclosure;

FIG. 4 depicts a flow diagram of an example method for determiningparameters of a classifier that produces a degree of association of atoken of a natural language text with a certain category of namedentities, in accordance with one or more aspects of the presentdisclosure;

FIG. 5 depicts a flow diagram of one illustrative example of a methodfor performing a semantico-syntactic analysis of a natural languagesentence, in accordance with one or more aspects of the presentdisclosure.

FIG. 6 schematically illustrates an example of a lexico-morphologicalstructure of a sentence, in accordance with one or more aspects of thepresent disclosure;

FIG. 7 schematically illustrates language descriptions representing amodel of a natural language, in accordance with one or more aspects ofthe present disclosure;

FIG. 8 schematically illustrates examples of morphological descriptions,in accordance with one or more aspects of the present disclosure;

FIG. 9 schematically illustrates examples of syntactic descriptions, inaccordance with one or more aspects of the present disclosure;

FIG. 10 schematically illustrates examples of semantic descriptions, inaccordance with one or more aspects of the present disclosure;

FIG. 11 schematically illustrates examples of lexical descriptions, inaccordance with one or more aspects of the present disclosure;

FIG. 12 schematically illustrates example data structures that may beemployed by one or more methods implemented in accordance with one ormore aspects of the present disclosure;

FIG. 13 schematically illustrates an example graph of generalizedconstituents, in accordance with one or more aspects of the presentdisclosure;

FIG. 14 illustrates an example syntactic structure corresponding to thesentence illustrated by FIG. 13;

FIG. 15 illustrates a semantic structure corresponding to the syntacticstructure of FIG. 14;

FIG. 16 depicts a diagram of an example computer system implementing themethods described herein.

DETAILED DESCRIPTION

Described herein are methods and systems for multi-stage recognition ofnamed entities in natural language texts based on morphological andsemantic features. The systems and methods described herein may beemployed in a wide variety of natural language processing applications,including machine translation, semantic indexing, semantic search(including multi-lingual semantic search), document classification,e-discovery, etc.

“Computer system” herein shall refer to a data processing device havinga general purpose processor, a memory, and at least one communicationinterface. Examples of computer systems that may employ the methodsdescribed herein include, without limitation, desktop computers,notebook computers, tablet computers, and smart phones.

Named entity recognition (NER) is an information extraction task thatlocates and classifies tokens in a natural language text intopre-defined categories such as names of persons, organizations,locations, expressions of times, quantities, monetary values,percentages, etc. Such categories may be represented by concepts of apre-defined or dynamically built ontology.

“Ontology” herein shall refer to a model representing objects pertainingto a certain branch of knowledge (subject area) and relationships amongsuch objects. An ontology may comprise definitions of a plurality ofclasses, such that each class corresponds to a concept of the subjectarea. Each class definition may comprise definitions of one or moreobjects associated with the class. Following the generally acceptedterminology, a class may also be referred to as a concept of theontology, and an object belonging to a class may also be referred to asan instance of the concept. An informational object definition mayrepresent a real life object (such as a person or a thing) or a certaincharacteristics associated with one or more real life objects (such as aquantifiable attribute or a quality). In certain implementations, aninformational object may be associated with two or more classes.

In certain implementations, the named entity recognition may beperformed based on a syntactico-semantic analysis of the naturallanguage text. However, such methods are characterized by a highcomputational complexity, and thus may not always be suitable forprocessing large corpora of natural texts, or in other situations wherethe computational complexity is at issue.

Systems and methods described herein improve the overall efficiency ofnamed entity recognition by performing a two-stage process, in which thefirst stage of the named entity recognition is performed based on alexico-morphological analysis of an input natural language text, whilethe second stage involving a syntactico-semantic analysis is optionallyperformed on at least part of the natural language text in situationswhen one or more metrics reflecting the quality of the first stage ofthe named entity recognition fall below respective thresholds. Since thelexico-morphological analysis is characterized by a lower computationalcomplexity in comparison with the syntactico-semantic analysis, thetwo-stage named entity recognition process described herein may be moreefficient than a single stage process involving a syntactico-semanticanalysis, e.g., when the second stage of the process is performed on arelatively small subset of the original text.

In accordance with one or more aspects of the present disclosure, thefirst stage of the named entity recognition involves alexico-morphological analysis of an input natural language text. Thelexico-morphological analysis may yield, for each sentence of thenatural language text, a lexico-morphological structure representing thesentence. Such a lexico-morphological structure may comprise, for eachword of the sentence, one or more lexical meanings and one or moregrammatical meanings of the word, which may be represented by one ormore <lexical meaning—grammatical meaning> pairs. A lexical meaning mayinclude one or more lemmas (i.e., canonical or dictionary forms)corresponding to the word, an identifier of a semantic class associatedwith the lexical meaning, and one or more classifying anddifferentiating semantemes. A grammatical meaning may be represented bya set of values of grammatical attributes, such as grammatical case,gender, number, conjugation type, aspect, tense, etc.

The lexical and grammatical attributes may then be fed to one or moreclassifier functions. Each classifier function may yield the degree ofassociation of the token with a certain category of named entities, thusconcluding the first stage of the named entity recognition process, asdescribed in more details herein below.

The computer system may then ascertain whether the second stage of thenamed entity recognition process should be performed. In an illustrativeexample, a graphical user interface (GUI) may be employed to render arepresentative fragment of the natural language text, while highlightingthe tokens representing recognized named entities. The computer systemmay accept the user input confirming or modifying the categoriesassociated with certain tokens, as well as the user input indicating thetokens and corresponding named entity categories that have not beenidentified by the first stage of the named entity recognition.Responsive to receiving the user input, the computer system maydetermine the accuracy and/or completeness of the first stage of thenamed entity recognition with respect to the representative textfragment. These quality metric values may be utilized in determiningwhether the second stage of the named entity recognition should beperformed, as described in more details herein below.

In certain implementations, the second stage of the named entityrecognition may be performed with respect to a certain subset of theoriginal natural language text, e.g., one or more sentences in which nonamed entities have been recognized or one or more sentences for which aquality metric falls below a certain threshold value, as described inmore details herein below.

The second stage of the named entity recognition may involvesyntactico-semantic analysis of the input natural language text. Thenthe computer system may employ a set of production rules to interpretthe semantic structures yielded by the syntactico-sematic analysis, thusproducing a plurality of data objects representing the identified namedentities, as described in more details herein below. The computer systemmay then extend a working ontology by associating the identified namedentities with the ontology classes corresponding to the named entitiescategories, as described in more details herein below.

Various aspects of the above referenced methods and systems aredescribed in details herein below by way of examples, rather than by wayof limitation.

FIG. 1 depicts a flow diagram of an example method for multi-stagerecognition of named entities in natural language texts based onmorphological and semantic features, in accordance with one or moreaspects of the present disclosure. Method 100 and/or each of itsindividual functions, routines, subroutines, or operations may beperformed by one or more processors of the computer system (e.g.,computer system 100 of FIG. 1) implementing the method. In certainimplementations, method 100 may be performed by a single processingthread. Alternatively, method 100 may be performed by two or moreprocessing threads, each thread implementing one or more individualfunctions, routines, subroutines, or operations of the method. In anillustrative example, the processing threads implementing method 100 maybe synchronized (e.g., using semaphores, critical sections, and/or otherthread synchronization mechanisms). Alternatively, the processingthreads implementing method 100 may be executed asynchronously withrespect to each other. Therefore, while FIG. 1 and the associateddescription lists the operations of method 100 in certain order, variousimplementations of the method may perform at least some of the describedoperations in parallel and/or in arbitrary selected orders.

At block 110, the computer system implementing method 100 may perform alexico-morphological analysis of an input natural language text 101,which may be represented, e.g., by one or more original documents. Thelexico-morphological analysis may yield, for each sentence of thenatural language text, a lexico-morphological structure 112 representingthe sentence. Such a lexico-morphological structure may comprise, foreach word of the sentence, one or more lexical meanings and one or moregrammatical meanings of the word, which may be represented by one ormore <lexical meaning—grammatical meaning> pairs. A lexical meaning mayinclude one or more lemmas (i.e., canonical or dictionary forms)corresponding to the word, an identifier of a semantic class associatedwith the word, and one or more classifying and differentiatingsemantemes. A grammatical meaning may be represented by a set of valuesof grammatical attributes, such as grammatical case, gender, number,conjugation type, aspect, tense, etc.

Non-dictionary words (such as named entities) may be associated with apre-defined semantic class (e.g., UNNKOWN). Grammatical meanings of anon-dictionary word may be determined by pseudo-lemmatization (i.e.,reconstructing a possible canonical form of the non-dictionary word),analysis of the context (e.g., two or more words surrounding thenon-dictionary word in a sentence), capitalization of one or moreletters of the non-dictionary word, etc.

An illustrative example of a method of performing lexico-morphologicalanalysis of a sentence is described in more details herein below withreferences to FIG. 6.

At block 115, the computer system may select, among the attribute valuesproduced by the lexico-morphological analysis, the attribute values thatwould be utilized as classifier function inputs. In certainimplementations, the highest ranking attribute values may be selected.The computer system may determine the attribute rating values based onone or more factors including statistical data on compatibility ofcertain lexemes and semantic classes, frequency of occurrence of aparticular lexical meaning in a corpus of natural language texts, etc.Alternatively, other methods of selecting the attribute values thatwould be utilized as classifier function inputs may be employed. In anillustrative example, all attribute values produced by thelexico-morphological analysis may be utilized as the classifier functioninputs. In an illustrative example, the attribute values may benormalized and/or processed by a certain convolution function in orderto reduce the dimension of the vector of attribute values.

At blocks 120-130, the computer system may iterate through a pluralityof tokens of the natural language text. At block 120, the computersystem may select the next token from the natural language text. A“token” herein shall refer to one or more adjacent words in a naturallanguage sentence.

At block 125, the computer system may determine, for each token, itsdegree of association with one or more categories of named entities. Thedegree of association of a token with a certain category of namedentities may, in an illustrative example, be represented by a realnumber selected from [0; 1] range.

For each category of named entities, one or more classifier functionsdefined in a hyperspace of the lexical and grammatical features may beemployed to determine the degree of association of an input naturallanguage token with a corresponding category of named entities. In anillustrative example, the following classifier functions may be employedfor the category Person: a first classifier function to recognize tokensreferencing persons by last names in natural language texts; a secondclassifier function to recognize natural text tokens referencing personsby first name and last name; and a third classifier function torecognize natural text tokens referencing persons by first name,optional middle name, and last name in different orders. In anotherillustrative example, the following classifier functions may be employedfor the category Location: a first classifier function to recognizenatural text tokens referencing countries; a second classifier functionto recognize natural text tokens referencing cities; and a thirdclassifier function to recognize natural text tokens referencing seasand rivers.

Building classifier functions and adjusting their parameters by applyingmachine learning methods is described in more details herein below.

For each token of a plurality of tokens of the natural language text, avector of values of the lexical and grammatical features characterizingthe token may be fed to one or more classifier functions to determinethe degree of association of the token with corresponding categories ofnamed entities, thus concluding the first stage of the named entityrecognition process.

Responsive to determining, at block 130, that the end of text has beenreached, the processing may continue at block 135; otherwise, the methodmay loop back to block 120.

At blocks 135-140, the computer system may ascertain whether the secondstage of the named entity recognition process on the text should beperformed. In certain implementations, the computer system may, at block135, evaluate one or more quality metrics characterizing the accuracyand/or completeness of the first stage of the named entity recognition.The accuracy of the named entity recognition may be characterized by theratio of the number of tokens that have been correctly associated withcategories and the number of tokens representing named entities thathave been identified in the natural language text. The completeness ofthe named entity recognition may be characterized by the ratio of thenumber of tokens that have been correctly associated with categories andthe total number of tokens representing named entities in the naturallanguage text.

In certain implementations, the accuracy and/or the completeness of thefirst stage of the named entity recognition may be evaluated based onthe user input received via a graphical user interface (GUI). In anillustrative example, a graphical user interface (GUI) may be employedto render a representative fragment of the natural language text, whilehighlighting the tokens representing recognized named entities, asschematically illustrated by FIG. 2. In an illustrative example, tokensassociated with each category of the named entities may be highlightedby a certain color. Alternatively, the category associated with acurrently selected token 210 and other relevant information may beprovided by a table 220 that may be displayed in a visual associationwith the fragment of the natural language text. The computer system mayaccept the user input confirming or modifying the categories associatedwith certain tokens, as well as the user input indicating the tokens andcorresponding named entity categories that have not been identified bythe first stage of the named entity recognition.

In certain implementations, responsive to receiving the user input, thecomputer system may determine the accuracy and/or completeness of thefirst stage of the named entity recognition with respect to therepresentative text fragment. These quality metric values may be assumedto represent the whole text being analyzed, and thus may be utilized indetermining whether the second stage of the named entity recognitionshould be performed. In an illustrative example, responsive todetermining, at block 140, that at least one quality metric value fallsbelow a corresponding pre-defined or dynamically adjusted threshold, theprocessing may continue at block 145; otherwise, the method mayterminate.

In certain implementations, the second stage of the named entityrecognition may be performed with respect to a certain subset of theoriginal natural language text, which may be identified at block 145. Inan illustrative example, the computer system may, at block 145, identifyone or more sentences in which no named entities have been recognized.In another illustrative example, the computer system may, at block 145,identify one or more sentences in which the quality metric falls below apre-defined or dynamically adjusted threshold value. The computer systemmay then perform the second stage of the named entity recognition withrespect to one or more text fragments comprising the identifiedsentences. In yet another illustrative example, the computer system may,at block 145, identify one or more sentences (e.g., sentences that onlycomprise dictionary words) to be excluded from the second stage of theanalysis.

At block 150, the computer system may initiate the second stage of thenamed entity recognition by performing syntactico-semantic analysis ofthe input natural language text 101. The syntactico-sematic analysis mayproduce language-independent semantic structures 152 representing thesentences of the natural language text, as described in more detailsherein below with references to FIGS. 5-15.

At block 155, the computer system may interpret the plurality ofsemantic structures using a set of production rules to produce aplurality of extracted objects representing the identified namedentities, as described in more details herein below.

In certain implementations, the extracted objects may be represented byan RDF graph 157. The Resource Definition Framework assigns a uniqueidentifier to each informational object and stores the informationregarding such an object in the form of SPO triplets, where S stands for“subject” and contains the identifier of the object, P stands for“predicate” and identifies some property of the object, and O stands for“object” and stores the value of that property of the object. This valuecan be either a primitive data type (string, number, Boolean value) oran identifier of another object. In an illustrative example, an SPOtriplet may associate a token of the natural language text with acategory of named entities.

The production rules employed for interpreting the semantic structuresmay comprise interpretation rules and identification rules. Aninterpretation rule may comprise a left-hand side represented by a setof logical expressions defined on one or more semantic structuretemplates and a right-hand side represented by one or more statementsregarding the informational objects representing the entities referencedby the natural language text.

A semantic structure template may comprise certain semantic structureelements (e.g., association with a certain lexical/semantic class,association with a certain surface or deep slot, the presence of acertain grammeme or semanteme etc.). The relationships between thesemantic structure elements may be specified by one or more logicalexpressions (conjunction, disjunction, and negation) and/or byoperations describing mutual positions of nodes within thesyntactico-semantic tree. In an illustrative example, such an operationmay verify whether one node belongs to a subtree of another node.

Matching the template defined by the left-hand side of a production ruleto a semantic structure representing at least part of a sentence of thenatural language text may trigger the right-hand side of the productionrule. The right-hand side of the production rule may associate one ormore attributes (reflecting lexical, syntactic, and/or semanticproperties of the words of an original sentence) with the informationalobjects represented by the nodes. In an illustrative example, theright-hand side of an interpretation rule may comprise a statementassociating a token of the natural language text with a category ofnamed entities.

An identification rule may be employed to associate a pair ofinformational objects which represent the same real world entity. Anidentification rule is a production rule, the left-hand side of whichcomprises one or more logical expressions referencing the semantic treenodes corresponding to the informational objects. If the pair ofinformational objects satisfies the conditions specified by the logicalexpressions, the informational objects are merged into a singleinformational object.

While in the illustrative example of FIG. 1 the second stage of thenamed entity recognition method employs, at block 155, production rules,various alternative implementations may employ classifier functionswhich may, along with lexical and morphological features, utilizesyntactic and/or semantic features produced by the lexico-semanticanalysis of the natural language text. In certain implementations, thenamed entity recognition method may employ a combination of productionrules and classifier models.

Referring again to FIG. 1, at block 160, the computer system may extenda working ontology 162 by associating the identified named entities withthe ontology classes corresponding to the named entities categories. Theworking ontology 162 may then be employed in a wide variety of naturallanguage processing applications, including machine translation,semantic indexing, semantic search (including multi-lingual semanticsearch), document classification, e-discovery, etc. Responsive tocompleting the operations referenced by block 160, the method mayterminate.

As noted herein above, in certain implementations, in estimating thedegree of association of a given natural language token with a categoryof named entities, the computer system may employ classifier functions.Certain parameters of such classifier functions may be adjusted bymachine learning methods that utilize pre-existing or dynamicallycreated evidence data sets. An evidence data set may comprise one ormore natural language texts, in which certain named entities and theirrespective categories are marked up. In an illustrative example, such anevidence data set may be created or updated by a GUI employed to accepta user input highlighting one or more adjacent words and associatingthem with a named entity category.

In certain implementations, the classifier function may be provided byan adaptive boosting (AdaBoost) with decision trees classifier. Adecision tree algorithm uses a decision tree as a predictive model tomap observed parameters of an item (e.g., lexical or grammaticalfeatures of a natural language token) to conclusions about the itemtarget value (e.g., a named entity category associated with the naturallanguage token). The method may operate on a classification tree inwhich each internal node is labeled with an input feature (e.g., lexicalor grammatical features of a natural language token). The edgesconnected to a node labeled with a feature are labeled with the possiblevalues of the input feature. Each leaf of the tree is labeled with anidentifier of a class (e.g., a named entity category associated with thenatural language token) or a degree association with the class.

In certain implementations, a training data set utilized by the machinelearning methods may comprise one or more of natural language texts, inwhich specified are named entity categories associated with certaintokens, as schematically illustrated by FIG. 3. In the example of FIG.3, opening and closing tags <PER> and </PER> are employed to delineatetokens that reference persons, opening and closing tags <LOC> and </LOC>are employed to designate tokens that reference locations, opening andclosing tags <EVENT> and </EVENT> are employed to designate tokens thatreference events, and opening and closing tags <DAY> and </DAY> areemployed to designate tokens that reference calendar dates. Variousother tags may be used to delineate tokens referencing objectsassociated with other categories of named entities.

In certain implementations, additional qualifying tags may be employedto define sub-categories of named entities. In an illustrative example,an object referenced by an <EVENT> tag may be further qualified, usingadditional tags, as a sporting event, anniversary, premiere performance,movies release, product launch, etc. In another illustrative example, anobject referenced by a <PER> tag may be further qualified, usingadditional tags, as a politician, celebrity, writer, artist, etc. In yetanother illustrative example, an object referenced by a <LOC> tag may befurther qualified, using additional tags, as a continent, country, city,capital, street, etc.

FIG. 4 depicts a flow diagram of an example method for determiningparameters of a classifier function that produces a degree ofassociation of a token of a natural language text with a certaincategory of named entities, in accordance with one or more aspects ofthe present disclosure. Method 400 and/or each of its individualfunctions, routines, subroutines, or operations may be performed by oneor more processors of the computer system (e.g., computer system 400 ofFIG. 4) implementing the method. In certain implementations, method 400may be performed by a single processing thread. Alternatively, method400 may be performed by two or more processing threads, each threadimplementing one or more individual functions, routines, subroutines, oroperations of the method. In an illustrative example, the processingthreads implementing method 400 may be synchronized (e.g., usingsemaphores, critical sections, and/or other thread synchronizationmechanisms). Alternatively, the processing threads implementing method400 may be executed asynchronously with respect to each other.Therefore, while FIG. 4 and the associated description lists theoperations of method 400 in certain order, various implementations ofthe method may perform at least some of the described operations inparallel and/or in arbitrary selected orders.

At block 410, the computer system implementing method 400 may perform alexico-morphological analysis of one or more natural language textsrepresenting the training set. The lexico-morphological analysis mayyield, for each sentence of the natural language text, alexico-morphological structure representing the sentence. Such alexico-morphological structure may comprise, for each word of thesentence, one or more lexical meanings and one or more grammaticalmeanings of the word, which may be represented by one or more <lexicalmeaning—grammatical meaning> pairs.

At block 420, the computer system may select, among the attribute valuesproduced by the lexico-morphological analysis, the attribute values thatwould be utilized as classifier function inputs. In certainimplementations, the highest ranking attribute values may be selected.The computer system may determine the attribute rating values based onone or more factors including statistical data on compatibility ofcertain lexemes and semantic classes, frequency of occurrence of aparticular lexical meaning in a corpus of natural language texts, etc.

In certain implementations, the computer system may resolve theambiguity arising from possible homonymy and/or coinciding grammaticalforms corresponding to different lexical meanings of a certain word,which may result in two or more <lexical meaning—grammatical meaning>pairs having been associated with such a word. In order to resolve theambiguity, the computer system may associate each <lexicalmeaning—grammatical meaning> pair with a rating, which may, in anillustrative example, be represented by a real number selected from [0;1] range. The computer system may determine the rating value based onone or more factors including statistical data on compatibility ofcertain lexemes and semantic classes, frequency of occurrence of aparticular lexical meaning in a corpus of natural language texts, etc.In certain implementations, among several <lexical meaning—grammaticalmeaning> pairs associated with a given word, the computer system mayselect a pair having the highest rank value.

Alternatively, other methods of selecting the attribute values thatwould be utilized as classifier function inputs may be employed. In anillustrative example, all attribute values produced by thelexico-morphological analysis may be utilized as the classifier functioninputs. In an illustrative example, the attribute values may benormalized and/or processed by a certain convolution function in orderto reduce the dimension of the vector of attribute values.

At block 430, the computer system may determine values of the classifierparameters that would optimize a chosen objective function (e.g.,maximize the accuracy and/or completeness of the named entityrecognition with respect to the training data set using the specifiedvalues of the classifier parameters), and the method may terminate.

FIG. 5 depicts a flow diagram of one illustrative example of a method200 for performing a semantico-syntactic analysis of a natural languagesentence 212, in accordance with one or more aspects of the presentdisclosure. Method 200 may be applied to one or more syntactic units(e.g., sentences) comprised by a certain text corpus, in order toproduce a plurality of semantico-syntactic trees corresponding to thesyntactic units. In various illustrative examples, the natural languagesentences to be processed by method 200 may be retrieved from one ormore electronic documents which may be produced by scanning or otherwiseacquiring images of paper documents and performing optical characterrecognition (OCR) to produce the texts associated with the documents.The natural language sentences may be also retrieved from various othersources including electronic mail messages, social networks, digitalcontent files processed by speech recognition methods, etc.

At block 214, the computer system implementing the method may performlexico-morphological analysis of sentence 212 to identify morphologicalmeanings of the words comprised by the sentence. “Morphological meaning”of a word herein shall refer to one or more lemmas (i.e., canonical ordictionary forms) corresponding to the word and a corresponding set ofvalues of grammatical attributes defining the grammatical value of theword. Such grammatical attributes may include the lexical category ofthe word and one or more morphological attributes (e.g., grammaticalcase, gender, number, conjugation type, etc.). Due to homonymy and/orcoinciding grammatical forms corresponding to differentlexico-morphological meanings of a certain word, two or moremorphological meanings may be identified for a given word. Anillustrative example of performing lexico-morphological analysis of asentence is described in more details herein below with references toFIG. 6.

At block 215, the computer system may perform a rough syntactic analysisof sentence 212. The rough syntactic analysis may include identificationof one or more syntactic models which may be associated with sentence212 followed by identification of the surface (i.e., syntactic)associations within sentence 212, in order to produce a graph ofgeneralized constituents. “Constituent” herein shall refer to acontiguous group of words of the original sentence, which behaves as asingle grammatical entity. A constituent comprises a core represented byone or more words, and may further comprise one or more childconstituents at lower levels. A child constituent is a dependentconstituent and may be associated with one or more parent constituents.

At block 216, the computer system may perform a precise syntacticanalysis of sentence 212, to produce one or more syntactic trees of thesentence. The pluralism of possible syntactic trees corresponding to agiven original sentence may stem from homonymy and/or coincidinggrammatical forms corresponding to different lexico-morphologicalmeanings of one or more words within the original sentence. Among themultiple syntactic trees, one or more best syntactic tree correspondingto sentence 212 may be selected, based on a certain rating functiontalking into account compatibility of lexical meanings of the originalsentence words, surface relationships, deep relationships, etc.

At block 217, the computer system may process the syntactic trees to theproduce a semantic structure 218 corresponding to sentence 212. Semanticstructure 218 may comprise a plurality of nodes corresponding tosemantic classes, and may further comprise a plurality of edgescorresponding to semantic relationships, as described in more detailsherein below.

FIG. 6 schematically illustrates an example of a lexico-morphologicalstructure of a sentence, in accordance with one or more aspects of thepresent disclosure. Example lexical-morphological structure 300 maycomprise a plurality of “lexical meaning-grammatical value” pairs forexample sentence. In an illustrative example, “ll” may be associatedwith lexical meaning “shall” 312 and “will” 314. The grammatical valueassociated with lexical meaning 312 is <Verb, GTVerbModal, ZeroType,Present, Nonnegative, Composite II>. The grammatical value associatedwith lexical meaning 314 is <Verb, GTVerbModal, ZeroType, Present,Nonnegative, Irregular, Composite II>.

FIG. 7 schematically illustrates language descriptions 210 includingmorphological descriptions 201, lexical descriptions 203, syntacticdescriptions 202, and semantic descriptions 204, and their relationshipthereof. Among them, morphological descriptions 201, lexicaldescriptions 203, and syntactic descriptions 202 are language-specific.A set of language descriptions 210 represent a model of a certainnatural language.

In an illustrative example, a certain lexical meaning of lexicaldescriptions 203 may be associated with one or more surface models ofsyntactic descriptions 202 corresponding to this lexical meaning. Acertain surface model of syntactic descriptions 202 may be associatedwith a deep model of semantic descriptions 204.

FIG. 8 schematically illustrates several examples of morphologicaldescriptions. Components of the morphological descriptions 201 mayinclude: word inflexion descriptions 310, grammatical system 320, andword formation description 330, among others. Grammatical system 320comprises a set of grammatical categories, such as, part of speech,grammatical case, grammatical gender, grammatical number, grammaticalperson, grammatical reflexivity, grammatical tense, grammatical aspect,and their values (also referred to as “grammemes”), including, forexample, adjective, noun, or verb; nominative, accusative, or genitivecase; feminine, masculine, or neutral gender; etc. The respectivegrammemes may be utilized to produce word inflexion description 310 andthe word formation description 330.

Word inflexion descriptions 310 describe the forms of a given worddepending upon its grammatical categories (e.g., grammatical case,grammatical gender, grammatical number, grammatical tense, etc.), andbroadly includes or describes various possible forms of the word. Wordformation description 330 describes which new words may be constructedbased on a given word (e.g., compound words).

According to one aspect of the present disclosure, syntacticrelationships among the elements of the original sentence may beestablished using a constituent model. A constituent may comprise agroup of neighboring words in a sentence that behaves as a singleentity. A constituent has a word at its core and may comprise childconstituents at lower levels. A child constituent is a dependentconstituent and may be associated with other constituents (such asparent constituents) for building the syntactic descriptions 202 of theoriginal sentence.

FIG. 9 illustrates exemplary syntactic descriptions. The components ofthe syntactic descriptions 202 may include, but are not limited to,surface models 410, surface slot descriptions 420, referential andstructural control description 456, control and agreement description440, non-tree syntactic description 450, and analysis rules 460.Syntactic descriptions 102 may be used to construct possible syntacticstructures of the original sentence in a given natural language, takinginto account free linear word order, non-tree syntactic phenomena (e.g.,coordination, ellipsis, etc.), referential relationships, and otherconsiderations.

Surface models 410 may be represented as aggregates of one or moresyntactic forms (“syntforms” 412) employed to describe possiblesyntactic structures of the sentences that are comprised by syntacticdescription 102. In general, the lexical meaning of a natural languageword may be linked to surface (syntactic) models 410. A surface modelmay represent constituents which are viable when the lexical meaningfunctions as the “core.” A surface model may include a set of surfaceslots of the child elements, a description of the linear order, and/ordiatheses. “Diathesis” herein shall refer to a certain relationshipbetween an actor (subject) and one or more objects, having theirsyntactic roles defined by morphological and/or syntactic means. In anillustrative example, a diathesis may be represented by a voice of averb: when the subject is the agent of the action, the verb is in theactive voice, and when the subject is the target of the action, the verbis in the passive voice.

A constituent model may utilize a plurality of surface slots 415 of thechild constituents and their linear order descriptions 416 to describegrammatical values 414 of possible fillers of these surface slots.Diatheses 417 may represent relationships between surface slots 415 anddeep slots 514 (as shown in FIG. 11). Communicative descriptions 480describe communicative order in a sentence.

Linear order description 416 may be represented by linear orderexpressions reflecting the sequence in which various surface slots 415may appear in the sentence. The linear order expressions may includenames of variables, names of surface slots, parenthesis, grammemes,ratings, the “or” operator, etc. In an illustrative example, a linearorder description of a simple sentence of “Boys play football” may berepresented as “Subject Core Object_Direct,” where Subject, Core, andObject_Direct are the names of surface slots 415 corresponding to theword order.

Communicative descriptions 480 may describe a word order in a syntform412 from the point of view of communicative acts that are represented ascommunicative order expressions, which are similar to linear orderexpressions. The control and concord description 440 may comprise rulesand restrictions which are associated with grammatical values of therelated constituents and may be used in performing syntactic analysis.

Non-tree syntax descriptions 450 may be created to reflect variouslinguistic phenomena, such as ellipsis and coordination, and may be usedin syntactic structures transformations which are generated at variousstages of the analysis according to one or more aspects of the presentdisclosure. Non-tree syntax descriptions 450 may include ellipsisdescription 452, coordination description 454, as well as referentialand structural control description 430, among others.

Analysis rules 460 may generally describe properties of a specificlanguage and may be used in performing the semantic analysis. Analysisrules 460 may comprise rules of identifying semantemes 462 andnormalization rules 464. Normalization rules 464 may be used fordescribing language-dependent transformations of semantic structures.

FIG. 10 illustrates exemplary semantic descriptions. Components ofsemantic descriptions 204 are language-independent and may include, butare not limited to, a semantic hierarchy 510, deep slots descriptions520, a set of semantemes 530, and pragmatic descriptions 540.

The core of the semantic descriptions may be represented by semantichierarchy 510 which may comprise semantic notions (semantic entities)which are also referred to as semantic classes. The latter may bearranged into hierarchical structure reflecting parent-childrelationships. In general, a child semantic class may inherits one ormore properties of its direct parent and other ancestor semanticclasses. In an illustrative example, semantic class SUBSTANCE is a childof semantic class ENTITY and the parent of semantic classes GAS, LIQUID,METAL, WOOD_MATERIAL, etc.

Each semantic class in semantic hierarchy 510 may be associated with acorresponding deep model 512. Deep model 512 of a semantic class maycomprise a plurality of deep slots 514 which may reflect semantic rolesof child constituents in various sentences that include objects of thesemantic class as the core of the parent constituent. Deep model 512 mayfurther comprise possible semantic classes acting as fillers of the deepslots. Deep slots 514 may express semantic relationships, including, forexample, “agent,” “addressee,” “instrument,” “quantity,” etc. A childsemantic class may inherit and further expand the deep model of itsdirect parent semantic class.

Deep slots descriptions 520 reflect semantic roles of child constituentsin deep models 512 and may be used to describe general properties ofdeep slots 514. Deep slots descriptions 520 may also comprisegrammatical and semantic restrictions associated with the fillers ofdeep slots 514. Properties and restrictions associated with deep slots514 and their possible fillers in various languages may be substantiallysimilar and often identical. Thus, deep slots 514 arelanguage-independent.

System of semantemes 530 may represents a plurality of semanticcategories and semantemes which represent meanings of the semanticcategories. In an illustrative example, a semantic category“DegreeOfComparison” may be used to describe the degree of comparisonand may comprise the following semantemes: “Positive,”“ComparativeHigherDegree,” and “SuperlativeHighestDegree,” among others.In another illustrative example, a semantic category“RelationToReferencePoint” may be used to describe an order (spatial ortemporal in a broad sense of the words being analyzed), such as beforeor after a reference point, and may comprise the semantemes “Previous”and “Subsequent.” In yet another illustrative example, a semanticcategory “EvaluationObjective” can be used to describe an objectiveassessment, such as “Bad,” “Good,” etc.

System of semantemes 530 may include language-independent semanticattributes which may express not only semantic properties but alsostylistic, pragmatic and communicative properties. Certain semantemesmay be used to express an atomic meaning which corresponds to a regulargrammatical and/or lexical expression in a natural language. By theirintended purpose and usage, sets of semantemes may be categorized, e.g.,as grammatical semantemes 532, lexical semantemes 534, and classifyinggrammatical (differentiating) semantemes 536.

Grammatical semantemes 532 may be used to describe grammaticalproperties of the constituents when transforming a syntactic tree into asemantic structure. Lexical semantemes 534 may describe specificproperties of objects (e.g., “being flat” or “being liquid”) and may beused in deep slot descriptions 520 as restriction associated with thedeep slot fillers (e.g., for the verbs “face (with)” and “flood,”respectively). Classifying grammatical (differentiating) semantemes 536may express the differentiating properties of objects within a singlesemantic class. In an illustrative example, in the semantic class ofHAIRDRESSER, the semanteme of <<RelatedToMen>> is associated with thelexical meaning of “barber,” to differentiate from other lexicalmeanings which also belong to this class, such as “hairdresser,”“hairstylist,” etc. Using these language-independent semantic propertiesthat may be expressed by elements of semantic description, includingsemantic classes, deep slots, and semantemes, may be employed forextracting the semantic information, in accordance with one or moreaspects of the present invention.

Pragmatic descriptions 540 allow associating a certain theme, style orgenre to texts and objects of semantic hierarchy 510 (e.g., “EconomicPolicy,” “Foreign Policy,” “Justice,” “Legislation,” “Trade,” “Finance,”etc.). Pragmatic properties may also be expressed by semantemes. In anillustrative example, the pragmatic context may be taken intoconsideration during the semantic analysis phase.

FIG. 11 illustrates exemplary lexical descriptions. Lexical descriptions203 represent a plurality of lexical meanings 612, in a certain naturallanguage, for each component of a sentence. For a lexical meaning 612, arelationship 602 to its language-independent semantic parent may beestablished to indicate the location of a given lexical meaning insemantic hierarchy 510.

A lexical meaning 612 of lexical-semantic hierarchy 510 may beassociated with a surface model 410 which, in turn, may be associated,by one or more diatheses 417, with a corresponding deep model 512. Alexical meaning 612 may inherit the semantic class of its parent, andmay further specify its deep model 512.

A surface model 410 of a lexical meaning may comprise includes one ormore syntforms 412. A syntform, 412 of a surface model 410 may compriseone or more surface slots 415, including their respective linear orderdescriptions 416, one or more grammatical values 414 expressed as a setof grammatical categories (grammemes), one or more semantic restrictionsassociated with surface slot fillers, and one or more of the diatheses417. Semantic restrictions associated with a certain surface slot fillermay be represented by one or more semantic classes, whose objects canfill the surface slot.

FIG. 12 schematically illustrates example data structures that may beemployed by one or more methods described herein. Referring again toFIG. 5, at block 214, the computer system implementing the method mayperform lexico-morphological analysis of sentence 212 to produce alexico-morphological structure 722 of FIG. 12. Lexico-morphologicalstructure 722 may comprise a plurality of mapping of a lexical meaningto a grammatical value for each lexical unit (e.g., word) of theoriginal sentence. FIG. 6 schematically illustrates an example of alexico-morphological structure.

Referring again to FIG. 5, at block 215, the computer system may performa rough syntactic analysis of original sentence 212, in order to producea graph of generalized constituents 732 of FIG. 12. Rough syntacticanalysis involves applying one or more possible syntactic models ofpossible lexical meanings to each element of a plurality of elements ofthe lexico-morphological structure 722, in order to identify a pluralityof potential syntactic relationships within original sentence 212, whichare represented by graph of generalized constituents 732.

Graph of generalized constituents 732 may be represented by an acyclicgraph comprising a plurality of nodes corresponding to the generalizedconstituents of original sentence 212, and further comprising aplurality of edges corresponding to the surface (syntactic) slots, whichmay express various types of relationship among the generalized lexicalmeanings. The method may apply a plurality of potentially viablesyntactic models for each element of a plurality of elements of thelexico-morphological structure of original sentence 212 in order toproduce a set of core constituents of original sentence 212. Then, themethod may consider a plurality of viable syntactic models and syntacticstructures of original sentence 212 in order to produce graph ofgeneralized constituents 732 based on a set of constituents. Graph ofgeneralized constituents 732 at the level of the surface model mayreflect a plurality of viable relationships among the words of originalsentence 212. As the number of viable syntactic structures may berelatively large, graph of generalized constituents 732 may generallycomprise redundant information, including relatively large numbers oflexical meaning for certain nodes and/or surface slots for certain edgesof the graph.

Graph of generalized constituents 732 may be initially built as a tree,starting with the terminal nodes (leaves) and moving towards the root,by adding child components to fill surface slots 415 of a plurality ofparent constituents in order to reflect all lexical units of originalsentence 212.

In certain implementations, the root of graph of generalizedconstituents 732 represents a predicate. In the course of the abovedescribed process, the tree may become a graph, as certain constituentsof a lower level may be included into one or more constituents of anupper level. A plurality of constituents that represent certain elementsof the lexico-morphological structure may then be generalized to producegeneralized constituents. The constituents may be generalized based ontheir lexical meanings or grammatical values 414, e.g., based on part ofspeech designations and their relationships. FIG. 13 schematicallyillustrates an example graph of generalized constituents.

At block 216, the computer system may perform a precise syntacticanalysis of sentence 212, to produce one or more syntactic trees 742 ofFIG. 12 based on graph of generalized constituents 732. For each of oneor more syntactic trees, the computer system may determine a generalrating based on certain calculations and a priori estimates. The treehaving the optimal rating may be selected for producing the bestsyntactic structure 746 of original sentence 212.

In the course of producing the syntactic structure 746 based on theselected syntactic tree, the computer system may establish one or morenon-tree links (e.g., by producing redundant path between at least twonodes of the graph). If that process fails, the computer system mayselect a syntactic tree having a suboptimal rating closest to theoptimal rating, and may attempt to establish one or more non-treerelationships within that tree. Finally, the precise syntactic analysisproduces a syntactic structure 746 which represents the best syntacticstructure corresponding to original sentence 212. In fact, selecting thebest syntactic structure 746 also produces the best lexical values 240of original sentence 212.

At block 217, the computer system may process the syntactic trees to theproduce a semantic structure 218 corresponding to sentence 212. Semanticstructure 218 may reflect, in language-independent terms, the semanticsconveyed by original sentence. Semantic structure 218 may be representedby an acyclic graph (e.g., a tree complemented by at least one non-treelink, such as an edge producing a redundant path among at least twonodes of the graph). The original natural language words are representedby the nodes corresponding to language-independent semantic classes ofsemantic hierarchy 510. The edges of the graph represent deep (semantic)relationships between the nodes. Semantic structure 218 may be producedbased on analysis rules 460, and may involve associating, one or moreattributes (reflecting lexical, syntactic, and/or semantic properties ofthe words of original sentence 212) with each semantic class.

FIG. 14 illustrates an example syntactic structure of a sentence derivedfrom the graph of generalized constituents illustrated by FIG. 13. Node901 corresponds to the lexical element “life” 906 in original sentence212. By applying the method of syntactico-semantic analysis describedherein, the computer system may establish that lexical element “life”906 represents one of the lexemes of a derivative form “live” 902associated with a semantic class “LIVE” 904, and fills in a surface slot$Adjunctr_Locative (905) of the parent constituent, which is representedby a controlling node $Verb:succeed:succeed:TO_SUCCEED (907).

FIG. 15 illustrates a semantic structure corresponding to the syntacticstructure of FIG. 14. With respect to the above referenced lexicalelement “life” 906 of FIG. 14, the semantic structure comprises lexicalclass 1010 and semantic classes 1030 similar to those of FIG. 14, butinstead of surface slot 905, the semantic structure comprises a deepslot “Sphere” 1020.

As noted herein above, and ontology may be provided by a modelrepresenting objects pertaining to a certain branch of knowledge(subject area) and relationships among such objects. Thus, an ontologyis different from a semantic hierarchy, despite the fact that it may beassociated with elements of a semantic hierarchy by certainrelationships (also referred to as “anchors”). An ontology may comprisedefinitions of a plurality of classes, such that each class correspondsto a concept of the subject area. Each class definition may comprisedefinitions of one or more objects associated with the class. Followingthe generally accepted terminology, an ontology class may also bereferred to as concept, and an object belonging to a class may also bereferred to as an instance of the concept.

In accordance with one or more aspects of the present disclosure, thecomputer system implementing the methods described herein may index oneor more parameters yielded by the semantico-syntactic analysis. Thus,the methods described herein allow considering not only the plurality ofwords comprised by the original text corpus, but also pluralities oflexical meanings of those words, by storing and indexing all syntacticand semantic information produced in the course of syntactic andsemantic analysis of each sentence of the original text corpus. Suchinformation may further comprise the data produced in the course ofintermediate stages of the analysis, the results of lexical selection,including the results produced in the course of resolving theambiguities caused by homonymy and/or coinciding grammatical formscorresponding to different lexico-morphological meanings of certainwords of the original language.

One or more indexes may be produced for each semantic structure. Anindex may be represented by a memory data structure, such as a table,comprising a plurality of entries. Each entry may represent a mapping ofa certain semantic structure element (e.g., one or more words, asyntactic relationship, a morphological, lexical, syntactic or semanticproperty, or a syntactic or semantic structure) to one or moreidentifiers (or addresses) of occurrences of the semantic structureelement within the original text.

In certain implementations, an index may comprise one or more values ofmorphological, syntactic, lexical, and/or semantic parameters. Thesevalues may be produced in the course of the two-stage semantic analysis,as described in more details herein. The index may be employed invarious natural language processing tasks, including the task ofperforming semantic search.

The computer system implementing the method may extract a wide spectrumof lexical, grammatical, syntactic, pragmatic, and/or semanticcharacteristics in the course of performing the syntactico-semanticanalysis and producing semantic structures. In an illustrative example,the system may extract and store certain lexical information,associations of certain lexical units with semantic classes, informationregarding grammatical forms and linear order, information regardingsyntactic relationships and surface slots, information regarding theusage of certain forms, aspects, tonality (e.g., positive and negative),deep slots, non-tree links, semantemes, etc.

The computer system implementing the methods described herein mayproduce, by performing one or more text analysis methods describedherein, and index any one or more parameters of the languagedescriptions, including lexical meanings, semantic classes, grammemes,semantemes, etc. Semantic class indexing may be employed in variousnatural language processing tasks, including semantic search,classification, clustering, text filtering, etc. Indexing lexicalmeanings (rather than indexing words) allows searching not only wordsand forms of words, but also lexical meanings, i.e., words havingcertain lexical meanings. The computer system implementing the methodsdescribed herein may also store and index the syntactic and semanticstructures produced by one or more text analysis methods describedherein, for employing those structures and/or indexes in semanticsearch, classification, clustering, and document filtering.

FIG. 16 illustrates a diagram of an example computer system 1000 whichmay execute a set of instructions for causing the computer system toperform any one or more of the methods discussed herein. The computersystem may be connected to other computer system in a LAN, an intranet,an extranet, or the Internet. The computer system may operate in thecapacity of a server or a client computer system in client-servernetwork environment, or as a peer computer system in a peer-to-peer (ordistributed) network environment. The computer system may be a providedby a personal computer (PC), a tablet PC, a set-top box (STB), aPersonal Digital Assistant (PDA), a cellular telephone, or any computersystem capable of executing a set of instructions (sequential orotherwise) that specify operations to be performed by that computersystem. Further, while only a single computer system is illustrated, theterm “computer system” shall also be taken to include any collection ofcomputer systems that individually or jointly execute a set (or multiplesets) of instructions to perform any one or more of the methodologiesdiscussed herein.

Exemplary computer system 1000 includes a processor 502, a main memory504 (e.g., read-only memory (ROM) or dynamic random access memory(DRAM)), and a data storage device 518, which communicate with eachother via a bus 530.

Processor 502 may be represented by one or more general-purpose computersystems such as a microprocessor, central processing unit, or the like.More particularly, processor 502 may be a complex instruction setcomputing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,or a processor implementing other instruction sets or processorsimplementing a combination of instruction sets. Processor 502 may alsobe one or more special-purpose computer systems such as an applicationspecific integrated circuit (ASIC), a field programmable gate array(FPGA), a digital signal processor (DSP), network processor, or thelike. Processor 502 is configured to execute instructions 526 forperforming the operations and functions discussed herein.

Computer system 1000 may further include a network interface device 522,a video display unit 510, a character input device 512 (e.g., akeyboard), and a touch screen input device 514.

Data storage device 518 may include a computer-readable storage medium524 on which is stored one or more sets of instructions 526 embodyingany one or more of the methodologies or functions described herein.Instructions 526 may also reside, completely or at least partially,within main memory 504 and/or within processor 502 during executionthereof by computer system 1000, main memory 504 and processor 502 alsoconstituting computer-readable storage media. Instructions 526 mayfurther be transmitted or received over network 516 via networkinterface device 522.

In certain implementations, instructions 526 may include instructions ofmethod 100 for determining confidence levels associated with attributevalues of informational objects, in accordance with one or more aspectsof the present disclosure. While computer-readable storage medium 524 isshown in the example of FIG. 16 to be a single medium, the term“computer-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “computer-readable storage medium” shall also betaken to include any medium that is capable of storing, encoding orcarrying a set of instructions for execution by the machine and thatcause the machine to perform any one or more of the methodologies of thepresent disclosure. The term “computer-readable storage medium” shallaccordingly be taken to include, but not be limited to, solid-statememories, optical media, and magnetic media.

The methods, components, and features described herein may beimplemented by discrete hardware components or may be integrated in thefunctionality of other hardware components such as ASICS, FPGAs, DSPs orsimilar devices. In addition, the methods, components, and features maybe implemented by firmware modules or functional circuitry withinhardware devices. Further, the methods, components, and features may beimplemented in any combination of hardware devices and softwarecomponents, or only in software.

In the foregoing description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that the present disclosure may be practicedwithout these specific details. In some instances, well-known structuresand devices are shown in block diagram form, rather than in detail, inorder to avoid obscuring the present disclosure.

Some portions of the detailed description have been presented in termsof algorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of operations leading to adesired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “determining,” “computing,” “calculating,” “obtaining,”“identifying,” “modifying” or the like, refer to the actions andprocesses of a computer system, or similar electronic computer system,that manipulates and transforms data represented as physical (e.g.,electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

The present disclosure also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Various other implementations will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the disclosure should, therefore, bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

What is claimed is:
 1. A method, comprising: performing, by a computersystem, a lexico-morphological analysis of a natural language textcomprising a plurality of tokens, each token comprising at least onenatural language word; determining, based on the lexico-morphologicalanalysis, one or more lexical meanings and grammatical meaningsassociated with each token of the plurality of tokens; for each token ofthe plurality of tokens, evaluating one or more classifier functionsusing the lexical and grammatical meanings associated with the tokens,wherein a value of each classifier function is indicative of a degree ofassociation of the token with a category of named entities; performing asyntactico-semantic analysis of at least part of the natural languagetext to produce a plurality of semantic structures representing the partof the natural language text; and interpreting the semantic structuresusing a set of production rules to determine, for one or more tokenscomprised by the part of the natural language text, a degree ofassociation of the token with a category of named entities.
 2. Themethod of claim 1, wherein the syntactico-semantic analysis of the partof the natural language text is performed responsive to determining thata metric indicative of a quality of associating the tokens withrespective categories falls below a threshold value.
 3. The method ofclaim 1, wherein the syntactico-semantic analysis of the part of thenatural language text is performed responsive to receiving a userinterface input.
 4. The method of claim 1, further comprising:identifying the part of the natural language text as comprising at leastone sentence in which no tokens have been associated with categories. 5.The method of claim 1, further comprising: identifying the part of thenatural language text as comprising at least one sentence in which ametric indicative of a quality of associating the tokens with respectivecategories falls below a threshold value.
 6. The method of claim 5,wherein the metric is provided by a ratio of a number of tokens thathave been correctly associated with categories to a number of tokensrepresenting named entities that have been identified in the naturallanguage text.
 7. The method of claim 5, wherein the metric is providedby a ratio of a number of tokens that have been correctly associatedwith categories to a total number of tokens representing named entitiesin the natural language text.
 8. The method of claim 1, wherein thesyntactico-semantic analysis of the part of the natural language text isperformed responsive to receiving a graphical user interface input. 9.The method of claim 1, further comprising: determining, using a trainingdata set, at least one parameter of the classifier function, wherein thetraining data set comprises a training natural language text comprisinga plurality of tokens, wherein each token is associated with a certaincategory.
 10. The method of claim 1, wherein each semantic structure ofthe plurality of semantic structures is represented by a graphcomprising a plurality of nodes corresponding to a plurality of semanticclasses and a plurality of edges corresponding to a plurality ofsemantic relationships.
 11. The method of claim 1, wherein a productionrule comprises one or more logical expressions defined on one or moresemantic structure templates.
 12. A system, comprising: a memory; aprocessor, coupled to the memory, the processor configured to: perform alexico-morphological analysis of a natural language text comprising aplurality of tokens, each token comprising at least one natural languageword; determine, based on the lexico-morphological analysis, one or morelexical meanings and grammatical meanings associated with each token ofthe plurality of tokens; for each token of the plurality of tokens,evaluate one or more classifier functions using the lexical andgrammatical meanings associated with the tokens, wherein a value of eachclassifier function is indicative of a degree of association of thetoken with a category of named entities; perform a syntactico-semanticanalysis of at least part of the natural language text to produce aplurality of semantic structures representing the part of the naturallanguage text; and interpret the semantic structures using a set ofproduction rules to determine, for one or more tokens comprised by thepart of the natural language text, a degree of association of the tokenwith a category of named entities.
 13. The system of claim 12, whereinthe syntactico-semantic analysis of the part of the natural languagetext is performed responsive to determining that a metric indicative ofa quality of associating the tokens with respective categories fallsbelow a threshold value.
 14. The system of claim 12, wherein thesyntactico-semantic analysis of the part of the natural language text isperformed responsive to receiving a user interface input.
 15. The systemof claim 12, wherein the processor is further configured to: identifythe part of the natural language text as comprising at least onesentence in which no tokens have been associated with categories. 16.The system of claim 12, wherein the processor is further configured to:identify the part of the natural language text as comprising at leastone sentence in which a metric indicative of a quality of associatingthe tokens with respective categories falls below a threshold value. 17.The system of claim 16, wherein the metric is provided by a ratio of anumber of tokens that have been correctly associated with categories toa number of tokens representing named entities that have been identifiedin the natural language text.
 18. The system of claim 16, wherein themetric is provided by a ratio of a number of tokens that have beencorrectly associated with categories to a total number of tokensrepresenting named entities in the natural language text.
 19. The systemof claim 12, wherein the processor is further configured to: determine,using a training data set, at least one parameter of the classifierfunction, wherein the training data set comprises a training naturallanguage text comprising a plurality of tokens, wherein each token isassociated with a certain category.
 20. A computer-readablenon-transitory storage medium comprising executable instructions that,when executed by a computer system, cause the computer system to:perform a lexico-morphological analysis of a natural language textcomprising a plurality of tokens, each token comprising at least onenatural language word; determine, based on the lexico-morphologicalanalysis, one or more lexical meanings and grammatical meaningsassociated with each token of the plurality of tokens; for each token ofthe plurality of tokens, evaluate one or more classifier functions usingthe lexical and grammatical meanings associated with the tokens, whereina value of each classifier function is indicative of a degree ofassociation of the token with a category of named entities; perform asyntactico-semantic analysis of at least part of the natural languagetext to produce a plurality of semantic structures representing the partof the natural language text; and interpret the semantic structuresusing a set of production rules to determine, for one or more tokenscomprised by the part of the natural language text, a degree ofassociation of the token with a category of named entities.